Collective Operations for Wide-Area Message Passing Systems Using Dynamically Created Spanning Trees

We propose a configuration-free method to perform collective operations efficiently in dynamically changing topologies. Our collective operations are designed so that (1) they perform well when the topology is stable, (2) they complete successfully even when processors join or leave, and (3) they adapt to topology changes. We propose to create adaptive latency-aware spanning trees for short messages and adaptive bandwidth-aware spanning trees for long messages, and perform collective operations along those trees. We implemented the latency-aware spanning tree on the Phoenix Message Passing Library, and confirmed that broadcasts and reductions execute faster than a topology-unaware implementation although not quite as fast a static topology-aware implementation. In another experiment, we studied the behavior of our broadcast when a part of the processors left and rejoined during computation—our broadcast temporarily performed worse while the spanning trees were being reconstructed, but completed successfully, and resumed effective execution once the topology change was over.